Skip to content

Comments

Checking parts of the message to combat flaky tests#1986

Merged
knative-prow[bot] merged 1 commit intoknative:mainfrom
cardil:bugfix/flaky-spinner
Jan 14, 2025
Merged

Checking parts of the message to combat flaky tests#1986
knative-prow[bot] merged 1 commit intoknative:mainfrom
cardil:bugfix/flaky-spinner

Conversation

@cardil
Copy link
Contributor

@cardil cardil commented Jan 13, 2025

Description

By checking for smaller parts of the expected TUI, we can effectively fix the flakiness, which comes primarily from race condition.

Changes

  • Checking parts of the message to combat flaky output/tui.TestSpinner

Reference

Fixes #1985

Copy link

@knative-prow knative-prow bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cardil: 0 warnings.

Details

In response to this:

Description

By checking for smaller parts of the expected TUI, we can effectively fix the flakiness, which comes primarily from race condition.

Changes

  • Checking parts of the message to combat flaky output/tui.TestSpinner

Reference

Fixes #1985

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@knative-prow knative-prow bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Jan 13, 2025
@cardil
Copy link
Contributor Author

cardil commented Jan 13, 2025

/assign @dsimansk

@cardil
Copy link
Contributor Author

cardil commented Jan 13, 2025

/cc @dsimansk

@knative-prow knative-prow bot requested a review from dsimansk January 13, 2025 19:30
@codecov
Copy link

codecov bot commented Jan 13, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 73.70%. Comparing base (c3b3be9) to head (82abc77).
Report is 2 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1986   +/-   ##
=======================================
  Coverage   73.70%   73.70%           
=======================================
  Files         234      234           
  Lines       16482    16482           
=======================================
  Hits        12148    12148           
  Misses       3506     3506           
  Partials      828      828           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@cardil
Copy link
Contributor Author

cardil commented Jan 13, 2025

I assured this will fix the issue by doing:

$ go test -c ./pkg/output/tui
$ ./tui.test -test.run=TestSpinner -test.count=10 -test.failfast
PASS
$ cpulimit -l 25 ./tui.test -test.run=TestSpinner -test.count=50 -test.failfast
PASS

On main, it was failing even on -test.count=1, when executed from pre-compiled binary.

Ref.: https://www.influxdata.com/blog/reproducing-a-flaky-test-in-go/

@cardil
Copy link
Contributor Author

cardil commented Jan 14, 2025

/test integration-tests

@dsimansk
Copy link
Contributor

/approve
/lgtm

@knative-prow knative-prow bot added the lgtm Indicates that a PR is ready to be merged. label Jan 14, 2025
@knative-prow
Copy link

knative-prow bot commented Jan 14, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cardil, dsimansk

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@knative-prow knative-prow bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 14, 2025
@dsimansk
Copy link
Contributor

Failing on a serving setup step is strange. Flake?

@dsimansk
Copy link
Contributor

/retest

@knative-prow knative-prow bot merged commit 8d3062d into knative:main Jan 14, 2025
23 checks passed
@cardil cardil deleted the bugfix/flaky-spinner branch January 14, 2025 14:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

output/tui.TestSpinner is flaky

2 participants